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REMARKS 

Applicant respectfully requests reconsideration and allowance of the 
subject application. Claims 5, 6, 19, 27, 28, 32, 33, 35, 36, 38 and 40-43 are 
amended. Claims 31, 39 and 45 are canceled without prejudice. New claims 
51-56 are added. Claims 1-30, 32-38, 40-44 and 46-56 are pending in this 
application. 

The Office Action indicates (p. 2) that the drawings are objected to 
because the reference characters 110 and 1 14 of p. 9 of the specification do not 
appear in the drawings. The amendments to the specification bring the 
drawings and specification into mutual conformance. No new matter is added 
by the amendment to the specification. Accordingly, Applicant respectfully 
requests that the objection to the drawing be withdrawn. 

The amendments to claims 6, 19 and 27 address minor informalities 
noted during review, however, these amendments are not intended to alter the 
scope of the claims. No new matter is added by these amendments. 

Claims 31, 39 and 45 have been canceled without prejudice in order to 
reduce the number of issues for consideration, claims 32 and 41 have been 
amended to place them in independent form and claims 33, 35, 36, 38, 40, 42 
and 43 have been amended to depend from claim 32 or 41. 

35 V.S.C. S 112 

Claims 5 and 28 stand rejected under 35 U.S.C. §112, second paragraph. 
Claims 5 and 28 have been amended to obviate the concems noted in the 
Office Action, however, these amendments are not intended to alter the scope 
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of the claims. Accordingly, Applicant respectfully requests that the §112 
rejections be withdrawn. 

35 U.S,C, S 103 

Claims 1-4, 9-45 and 50 stand rejected under 35 U.S.C. §103(a) as 
being unpatentable over U.S. Patent No. 5,893,095 to Jain et al. (hereinafter 
"Jain") in view of U.S. Patent No. 6,347,313 to Ma et al. (hereinafter "Ma"). 
Claims 5-8 and 46-49 stand rejected under 35 U.S.C. 103(a) as being 
unpatentable over Jain and Ma as applied above and further in view of 
"Relevance Feedback: A Power Tool for Interactive Content-Based Image 
Retrieval" by Rui et al., IEEE Trans, on Circuits and Systems for Video 
Technology, Vol. 8, No. 5, Sept. 1998 (IEEE Cat. No. 1051-8215/98). Claims 
31, 39 and 45 have been canceled without prejudice, rendering their rejection 
moot. 

The Examiner admits (p. 4) that Jain fails to teach the use of relevance 
feedback or user feedback to generate new query vectors. The Examiner finds 
suggestion for such in the discussion of query refinement or systematic 
browsing. Applicant respectfully disagrees, at least because such does not 
suggest relevance feedback. 

For example, neither query refinement nor systematic browsing 
corresponds to "no opinion", "irrelevant" or "highly irrelevant", as recited in 
new claims 51, 53 and 55. Either technique presumes and relies upon a degree 
of relevancy associated with either the query or whatever subject matter is 
being browsed. 



Iee®hayes poc 509-324-9256 



Application No. 09/660,536 



0 



Assuming arguendo that a user of Jain's system could manually generate 
a new query by the methods discussed in the Office Action still does not teach, 
disclose, suggest or motivate use of relevance or lack thereof in a recursive 
search scheme as recited in Applicant's claims. Suggestion to modify as put 
forth in the Office Action appears to employ an improper "obvious to try" 
rationale, as is discussed below in more detail with reference to MPEP 
§2145(X)(B). This MPEP section states that: 

The admonition that 'obvious to try' is not the standard under 
§103 has been directed mainly at two kinds of error. In some 
cases, what would have been 'obvious to try' would have been to 
vary all parameters or try each of numerous possible choices 
until one possibly arrived at a successful result, where the prior 
art gave either no indication of which parameters were critical or 
no direction as to which of many possible choices is likely to be 
successful.... In others, what was 'obvious to try' was to explore 
a new technology or general approach that seemed to be a 
promising field of experimentation, where the prior art gave only 
general guidance as to the particular form of the claimed 
invention or how to achieve it. In re OTarrell, 853 F.2d 894, 
903, 7 USPQ2d 1673, 1681 (Fed. Cir. 1988) (citations omitted). 

In fact, Jain employs terminology related to relevance exactly five 
times: (i) at col. 13, lines 64 and 65, noting that color is not relevant to analysis 
of X-ray images; (ii) with respect to technology relevant to image analysis (col. 
15, lines 4 and 5); (iii) to describe a relationship between "primitives" and 
image properties (col. 15, lines 58 and 59); (iv) to indicate a set of candidate 
images (col. 17, lines 28 and 29); and to indicate technology relevant to image 
analysis (col. 22, lines 27 and 28). Jain clearly understands the term "relevant" 
and chooses not to teach, disclose, suggest or motivate use of relevance in 
refining search criteria. 



tee@hayes puc 509-32*-9256 



Application No. 09/660,536 



The Examiner indicates (p. 4) that Ma discloses a system and method 
for retrieving objects based on indications of "relevance" and "irrelevance". 
The Examiner also acknowledges (p. 4) that Ma fails to teach, disclose, suggest 
or motivate generation of new query vectors based on relevance feedback. The 
Examiner suggests (p. 5) that motivation would flow from the teachings of Jain 
and the ability of Ma's system. 

Ma teaches a system for indexing and retrieving database objects that 
are grouped into clusters and employs a correlation matrix 12 to track 
correlations between clusters of such objects (see, e.g.. Abstract). Ma teaches 
updating of the correlation matrix and the clusters and does not teach, disclose, 
suggest or motivate modification of the query vectors or objects, or doing so in 
response to user feedback. 

Applicant notes that Jain fails to suggest use of relevance feedback. 
Further, even if the teachings of the references were somehow combined, 
despite a lack of any teachings in the references of how to do so, the result 
clearly fails to provide the invention as recited in any of Applicant's claims 1, 
11, 19, 32, 41 and 50 because the combination fails to provide, suggest or 
motivate generation of new query vectors based on relevance feedback or 
weighting feature elements based on relevance feedback, as recited, in varying 
language, in these claims. 

Claim 1 1 recites a method of selecting between two types of matrixes to 
be used to weight a plurality of feature elements. Jain is silent with respect to 
selection between types of matrixes, as is the Office Action. In fact, Jain is 
void of the word "matrix" or any other word sharing a root with such and thus 
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cannot possibly teach or disclose such. As a result, it is inconceivable that Jain 
could suggest or motivate the invention as recited in claim 1 1 . 

Ma teaches a system for indexing and retrieving database objects that 
are grouped into clusters and employs a correlation matrix 12 to track 
correlations between clusters of such objects (see, e.g.. Abstract). As such. Ma 
cannot possibly cure the deficiencies of Jain, and combining the teachings of 
Jain and Ma cannot provide the invention as recited in claim 1 1 . 

Claim 23 recites identifying a number of training samples, and, based on 
comparison of the number to a threshold, either transforming the query vector 
and the feature vector to a higher-level feature space or computing distances 
between the query vector and the feature vector. Claim 30 recites a system 
including a comparator configured to identify a number of training samples for 
which relevance feedback has been received, and which carries out analogous 
transformation and distance computations. 

Jain uses the word "transform" exactly once, to describe a 
transformation from the data-rich representation of explicit image pixels to a 
compact, semantic-rich representation of visually salient characteristics (see 
Summary). Accordingly, Jain does not teach or disclose the invention as 
recited in claims 23 and 30 and cannot and does not suggest or motivate such. 

Ma teaches transformation of feature vectors using modified radial basis 
functions (col. 6, line 23 et seq.). Ma is silent with respect to training samples, 
as is Jain. In fact, Jain and Ma are both void of the word "train" in any form. 
As a result, it is impossible to provide the invention as recited in claims 23 and 
30 by combining the teachings of Jain and Ma. 
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Claim 10 recites specific aspects of an image included in the low level 
features. Specifically, claim 10 recites that "the low-level features include: a 
color moments feature, a wavelet based texture feature, and a water-fill edge 
feature", which is not taught, disclosed, suggested or motivated by Jain or Ma. 

The Office Action states that such would be obvious. However, this is 
not the test for unpatentability (see infra). 

In fact, both Jain and Ma are void of the terms "color moment", 
"wavelet" or "water-fiU" and thus neither reference teaches or discloses such. 
As such, it is inconceivable that Jain or Ma, or any proper combination thereof, 
could render the invention as recited in claim 10 unpatentable. For at least 
these reasons, the rejection of claim 10 is improper and should be withdrawn, 
and claim 10 should be allowed. 

Claims 7 and 46 recite specific aspects of generation of a query vector 
incorporating user feedback, while claims 8 and 48 recite specific aspects of 
generating weights for distances between a query vector and a feature vector 
by incorporating relevance. The Office Action repeatedly admits that Rui does 
not teach these aspects of Applicants* claimed invention (pp. 12-15), and states 
that the alternatives described in the reference "have the same effect" as the 
invention as recited in Applicants' claims. This is not a relevant test for a 
finding of unpatentability, as noted below. Accordingly, Applicant 
respectfully requests that the unpatentability rejection based on Jain, Ma and 
Rui be withdrawn. 

Applicant notes the requirements of MPEP §2143, entitled "Basic 
Requirements of a Prima Facie Case of Obviousness" (see also MPEP 
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§706.02G), entitled "Contents of a 35 U.S.C. 103 Rejection.") . MPEP §2143 
states that "To establish a prima facie case of obviousness, three basic criteria 
must be met. First, there must be some suggestion or motivation, either in the 
references themselves or in the knowledge generally available to one of 
ordinary skill in the art, to modify the reference or to combine reference 
teachings." 

Inasmuch as the references fail to teach or disclose the elements recited 
in the claims, the references cannot provide motivation to modify their 
teachings to arrive at the invention as claimed, and the Examiner has identified 
no such teaching or disclosure in the references. As a result, the first prong of 
the test cannot be met. 

MPEP §2143 further states that "Second, there must be a reasonable 
expectation of success. Finally, the prior art reference (or references when 
combined) must teach or suggest all the claim limitations." 

Inasmuch as the references fail to provide all of the features recited in 
Applicant's claims, the third prong of the test is not met. As a result, there 
cannot be a reasonable expectation of success. As such, the second prong of 
the test cannot be met. 

MPEP §2143 additionally states that "The teaching or suggestion to 
make the claimed combination and the reasonable expectation of success must 
both be found in the prior art, not in applicant's disclosure. In re Vaeck, 947 
F.2d 488, 20 USPQ2d 1438 (Fed. Cir. 1991)." This criterion cannot be met 
because the references fail to teach or disclose the elements recited in the 
claim. Accordingly, the unpatentability rejections based on Jain as modified in 
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view of Ma and on Jain, Ma and Rui fail all of the criteria for establishing a 
prima facie case of obviousness as set forth in the MPEP. For at least these 
reasons. Applicant respectfully requests that the §103 rejections be withdrawn, 
and that Applicant's claims 1-30, 32-38, 40-44 and 46-56 be allowed. 

The cited references do not teach, disclose, suggest or motivate "One or 
more computer readable media including a computer program that is 
executable by a processor to cause the processor to perform acts of: receiving 
feedback regarding the relevance of each image of a set of images, the user 
feedback forming a range including at least Highly Relevant, Relevant, No 
Opinion, Irrelevant, and Highly Irrelevant; wherein represents the number 
of images in the set of images for which user relevance feedback has been 
received, represents the relevance of image n in the set of images, k 
represents a transposition of a vector generated by concatenating the individual 
values, and X represents an image matrix that is generated by stacking 'N 
training vectors corresponding to the set of images into a matrix; and 
generating a query vector ( ^ ) corresponding to one of a plurality of features as 
follows: 




ft 



, as recited in new claim 55. Therefore, Applicant believes 



that new claim 55 is allowable. 
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Conclusion 

Claims 1-30, 32-38, 40-44 and 46-56 are in condition for allowance. 
Applicant respectfully requests reconsideration and issuance of the subject 
application. Should any matter in this case remain unresolved, the undersigned 
attomey respectfully requests a telephone conference with the Examiner to 
resolve any such outstanding matter. 



Respectfully Submitted, 



Date: Cl^'-^t^^ 




Frederick M. Fliegel 
Reg. No. 36,138 
(509) 324-9256 x239 
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Version with Markings to Show Changes Made 
In the Specification: 

The paragraph on page 9 spanning Unes 3-6 has been amended as 
shown below: 

Computer 142 also includes a broadcast tuner 200. Broadcast tuner 
200 receives broadcast signals either directly (e.g., analog or digital cable 
transmissions fed directly into tuner 200) or via a reception device (e.g., via 
an antenna [1 10] or satellite dish [114 of Fig. 1]). 

In the Claims: 

5. (Amended) One or more computer readable media as recited in 
claim 2, wherein X represents an image matrix that is generated by 
stacking feature vectors , each of length corresponding to the set of 
potentially relevant images for which relevance feedback was received and 
resulting in an ( x matrix, C represents a weighted covariance matrix 
of X , det(C) represents the matrix determinant of C, and the matrix 
comprises a full matrix (fT * ) that is generated as follows: 
j_ 

^* = (det(C))^C-^ 
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6. (Amended) One or more computer readable media as recited in 
claim 2, wherein represents the kli^ element of matrix W , Xj^ represents 
the 1^^ feature element, represents the standard deviation of the sequence 
of *s, the matrix comprises a diagonal matrix with each diagonal element 
( ) being generated as follows: 
1 



19. (Amended) One or more computer readable media having 
stored thereon a plurality of instructions that, when executed by one or 
more processors, causes the one or more processors to perform acts 
including: 

comparing, for each of a plurality of images, a plurality of feature 
elements from a query vector to a plurality of feature elements from a 
feature vector corresponding to the image; 

identifying a number of potentially relevant images based on the 
comparing; 

receiving user feedback regarding [the] relevancy of one or more of 
the potentially relevant images; 

re-comparing, for each of the plurality of images, the plurality of 
feature elements from the query vector to the plurality of feature elements 
from the feature vector, including using a matrix to compare the feature 
elements and dynamically selecting a type of matrix to use based on both 
the user feedback and the number of the plurality of feature elements; 
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identifying a new set of potentially relevant images based on the re- 
comparing; and 

presenting the new set of potentially relevant images to the user. 

27. (Amended) A method as recited in claim 23, further 
comprising: 

repeating the generating, identifying of the feature vector, 
identifying of the number of training samples, and the determining for each 
of a plurality of features; and 

identifying how closely the image and the [other] another image 
match each other by combining the distances between the query vectors and 
the feature vectors for the plurality of features. 

28. (Amended) A method as recited in claim 27, wherein the 
identifying how closely the image and the another image match each other 
comprises calculating a weighted summation of each of the individual 
distances for each of the plurality of features. 

Please cancel claim 3 1 without prejudice. 



]ee@hayes poc 509-324-9256 



33 



Application No. 09/660.536 



32. (Amended) [A method as recited in claim 31] 
A method comprising: 

for one of a plurality of images and each of a plurality of features, 
generating, based on the set of search criteria, a query vector for the 
feature, 

identifying a feature vector, corresponding to the image, for the 
feature, and 

determining how closely the feature vector matches the query vector: 

and 

determining how closely the image matches the set of search criteria 
based on how closely, for the plurality of features, the feature vectors match 
the query vectors , wherein generating the query vector comprises 
generating the query vector based at least in part on user relevance 
feedback regarding how relevant images previously displayed to a user 
were. 

33, (Amended) A method as recited in claim [31] 32, wherein 
identifying the feature vector comprises: 

identifying a low-level feature vector corresponding to the feature; 

and 

mapping the low-level feature vector to a higher level feature space. 

35. (Amended) A method as recited in claim [31] 32, wherein the 
initial search criteria comprises an image. 
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36. (Amended) A method as recited in claim [31] 32, wherein the 
determining how closely the feature vector matches the query vector 
comprises determining a distance between the feature vector and the query 
vector, and wherein the determining how closely the image matches the set 
of search criteria comprises calculating a weighted summation of each of 
the individual distances between the feature vectors and the query vectors. 

37. A method as recited in claim 36, wherein the calculating a 
weighted summation comprises calculating the weighted summation based 
at least in part on user relevance feedback regarding how relevant images 
previously displayed to a user were. 

38. (Amended) One or more computer readable media including a 
computer program that is executable by a processor to perform the method 
recited in claim [31] 32. 

Please cancel claim 39 without prejudice. 

40. (Amended) One or more computer readable media as recited in 
claim [39] 41, wherein the identifying the plurality of query vectors 
comprises extracting the plurality of query vectors from the image. 
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41. (Amended) [One or more computer readable media as recited in 
claim 39] 

One or more computer readable media having stored thereon a 
plurality of instructions that, when executed by one or more processors, 
causes the one or more processors to perform acts including: 

identifying a plurality of query vectors for one image, each query 
vector corresponding to one of a plurality of features; 

identifying a plurality of feature vectors for another image, each 
feature vector corresponding to one of the plurality of features; 

for each feature, determining a distance between the corresponding 
query vector and the corresponding feature vector; and 

combining the distances to generate a value representing an overall 
distance between the one and the another image , wherein the identifying the 
plurality of query vectors comprises generating the plurality of query 
vectors based at least in part on user relevance feedback regarding how 
relevant images previously displayed to a user were. 

42. (Amended) One or more computer readable media as recited in 
claim [39] 41, wherein the determining the distance between the 
corresponding query vector and the corresponding feature vector includes 
incorporating, into the determining, user relevance feedback regarding how 
relevant images previously displayed to a user were. 
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43. (Amended) One or more computer readable media as recited in 
claim [39] 41, wherein the combining the distances comprises calculating a 
weighted summation of each of the individual distances between the feature 
vectors and the query vectors. 

Please cancel claim 45 without prejudice. 

Claims 51-56 have been added. 

END OF DOCUMENT 
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